test: count a check and record it in one operation (#917) - #923

…s the shell (#916)

Both from OffgridwithJD's review of #922, and both are real.

AN ABSENT FILE IS NOT AN EXEMPT SUITE. pgc_suite_declares_accounting returned
"no" for a file that does not exist as readily as for one that does not call
pgc_summary. A registered suite whose .sh had vanished was therefore classified
exempt, and the reconciliation read clean -- a suite disappearing from the matrix,
inside the check whose whole subject is suites going missing from the accounting.

It now answers "absent", the runner counts those and fails the major, and the
real-population block in selftest 390 carries three buckets rather than folding
absent back into exempt. Measured today: 251 registered, 0 absent.

THE COMMENT STRIPPER NOW FOLLOWS THE SHELL'S RULE. `sed 's/#.*$//'` strips from
ANY hash, so a hash inside a quoted string earlier on the line would hide a
pgc_summary call after it. It now strips only a hash at line start or after
whitespace, which is what the shell treats as starting a comment.

Measured rather than assumed, over all 251 registered suites: three carry a line
holding both a hash and pgc_summary -- analyze_function, hilbert_curve and
projections -- and in every one the hash starts the line, so no suite was misread
either way. The partition is 239/12 before and after.

The shell rule still does not cover a hash after whitespace INSIDE a quoted
string, so that residual is pinned by an arm over the corpus rather than left to
be rediscovered: no registered suite may have a hash before a pgc_summary call on
the same line.

New arms in both harnesses: absent is distinguishable from exempt, a hash inside
a word does not hide the call after it, a trailing comment does not either, an
indented comment is still a comment, and every registered suite has a file.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Unuuvh3fRR67SceiGpfeeK

…he twelve are (#916)

All from OffgridwithJD's review, including two corrections to claims I had
already pushed.

THE CORPUS ARM I ADDED LAST COMMIT WAS INVERTED. It required a non-whitespace
character before the hash, which is a hash inside a WORD -- the shape the
stripper handles correctly -- so it flagged the safe case and was blind to the
dangerous one it was named for. Reproduced here with their control:

    psql -c "SELECT 1 # note"; pgc_summary    reader says NO, arm did not flag
    X=a#b; pgc_summary                        reader says YES, arm FLAGGED

The arm no longer restates the hazard as a second pattern. It compares the
reader's INPUT with its OUTPUT: count the call in the raw file, count it in the
stripped text, and a lower stripped count means the stripper hid a call. That
catches it for any spelling, cannot be inverted, and IS the measurement rather
than depending on one staying true.

THE READER HAD NEVER BEEN SHOWN THE PRODUCER'S OWN OUTPUT. Every log in both
harnesses was a literal, and the format string lives a third time in
pgc_summary. Three hand-written copies of one line: a wording drift in the
PRODUCER leaves both harnesses green while the reader answers "no" for every real
suite, which would redden the whole matrix on both majors having passed its own
tests. Both harnesses now run a real two-line suite and feed the reader its
actual stdout, with a reworded control so the arm can fail.

THE TWELVE, MEASURED. My prose said ten suites "never counted a check" and my
partition said twelve; both reviewers then compounded it, because a loose pattern
for sourcing lib.sh matches portlib.sh. Measured with a tight one: NONE of the
twelve sources test/lib.sh. Each defines its own check(), and ten keep no tally at
all. The number is no longer written in prose anywhere; the reconciliation prints
it at runtime.

THE OVERCOUNT THE PR OPENS BY DESCRIBING IS NOW ACTUALLY REPORTED. #922's own CI
showed "suites that ran: 242 of 251" still counting all twelve suites whose checks
the harness cannot see -- so the change described a fix it did not make. The
summary now breaks them out, from data already in hand at that point:

    suites that ran: 242 of 251 (skipped: 9, incomplete: 0)
    of those, N accounted for their checks and M did not

Also: an absent-file answer the case now handles with a loud default arm rather
than folding an unknown verdict into "does not declare"; a sentence on why the
log reader deliberately does NOT distinguish absent from negative while the
declaration reader does; a precise note that inputs == sum(buckets) cannot be
false on the DATA -- for sets the identity always holds, measured over 400 random
pairs -- and that what it guards is comm reading unsorted input; and ci.yml's
"three wall-clock suites", which is stale at four and which this change is the one
to falsify.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Unuuvh3fRR67SceiGpfeeK

…s it (#486)

#923's `suites (PG 17)` went red on a pytest name that EXISTS, while PG 18 passed
the same commit:

    FAIL  every test the document names exists in the corpus:
          got [[1: test_the_partition_over_the_registered_suites_adds_up]]

The cause is the shape selftest 080 already forbids. Its membership test was
`printf '%s\n' "$ondisk" | grep -qxF "$name"`, and the selftest runs under
`set -o pipefail`: grep -q exits the moment it matches, printf takes EPIPE, and
pipefail reports the pipeline failed though the name WAS present. The name is
then recorded absent.

WHY IT SURVIVED, AND WHY IT SURFACED NOW. Selftest 080's sweep is deliberately
non-recursive and never looked inside test/selftest/ -- the directory scoping was
never a decision, it fell out of writing "$TESTDIR"/*.sh, exactly as the bench/
hole did before it. Three fragments held the forbidden shape: 350's corpus
membership test, 300's directory coverage test, 340's Makefile sweep.

At corpus size the writer is small enough to win the race on an idle machine,
which is why it passed for so long. Measured, 170 names over 400 trials:

    printf | grep -qxF     idle: 0 false absences    under load: 6
    grep -cxF <<<          idle: 0                   under load: 0

A four-way CI matrix is the loaded case, and adding names to the corpus made the
window wider. This is the third appearance of this bug class here after #473 and
#476, and the second today.

All three sites now use grep -c on a here-string, and the sweep covers
test/selftest/ with its own coverage arm, because 080 already records that a
conditionally added glob narrows silently and a file-count premise cannot see it.

THE EXEMPTION IS DERIVED, NOT LISTED. 080's own control is inside a quoted
heredoc, and so is any deliberate demonstration of the shape; a line inside one is
text being written to a file, not a pipeline the suite runs. Comment lines are
excluded for the same reason -- the rule's explanation, and the note beside each
site fixed here, necessarily spell the shape out. A filename allowlist would have
to be maintained, and this rule exists because things that must be maintained are
not.

Two defects in the exemption itself, both found by running it rather than reading
it. It printed NR where it meant FNR, so from the second file onwards it reported
line numbers from a running total and no key matched -- the neighbouring question
answered plausibly, since single-file runs agree because NR == FNR there. And it
kept heredoc state across inputs, so it now resets per file. Its own arms pin
both: the sweep sees both lines of a probe, the exemption covers the one inside
the heredoc and not the one above it.

The "did not swallow the corpus" premise is a PROPORTION rather than a guessed
ceiling. The first version used a bare 2000 and went red at 2,502 heredoc lines in
a corpus that was entirely healthy -- a hand-written number failing the way
hand-written numbers fail here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Unuuvh3fRR67SceiGpfeeK

…d printf (#486)

Reported by OffgridwithJD against the previous commit: widening the DIRECTORIES
left the PRODUCER scoped to echo and printf, so every `<command> | grep -q` was
still unswept. Twenty-six sites, two of them in lib.sh and shared by every suite
that asks whether a plan is a columnar scan.

THE SCOPING REVERSAL, AND WHY. The rule's own stated principle is a reader that
exits early AND whose EXIT STATUS is the answer being read. That is
producer-independent: the writer takes EPIPE whether it is a builtin, a psql, an
ldd or an ss. Scoping to echo and printf was a narrower implementation than the
principle, justified in the file by "a pipeline out of psql or a file is a
different question" -- which is true of `| head -1` used as TEXT and false of
`| grep -q` used as a VERDICT.

THE WORST SITE IS VACUITY, NOT A FALSE RED. native_vecskip.sh's "premise: and it
is not the scalar scan" WANTS "no", so a spurious EPIPE answer makes that premise
pass for the wrong reason. It is fixed first for that reason.

NOT CLAIMED TO BE LYING TODAY. Measured, 200 trials per size on a loaded box,
match always on line one so the answer is knowably yes:

    1,892 bytes (EXPLAIN-sized)    0/200 wrong
    8,893 bytes                    1/200   <- first observed lie
   66,894 bytes                   21/40
  288,894 bytes                   40/40
  control, match on the LAST line so grep reads to EOF: 0/200 at every size

Latent, with no floor in the mechanism -- the probability rises with size rather
than crossing a threshold, which refuted a clean pipe-capacity hypothesis.
Reasoning about "small enough" is how #473, #476 and selftest 350 each survived,
so the rule sweeps rather than reasons.

`||` IS NOT A PIPE, and the first version of the widened pattern thought it was:
`[ "$rc" = 124 ] || grep -q PAT <<<"$out"` is a fallback branch reading a
here-string, with no writer process and so no EPIPE, and both fuzz suites were
flagged for it. The leading [^|] excludes it, with an arm pinning that.

My own deliberate twin in selftest 390 moved into a quoted heredoc, so the
widened sweep exempts it by the property already built rather than by a line
number -- a filename list is the thing this rule exists to avoid.

THE MATRIX IS THE VERIFICATION HERE. lib.sh's pgc_is_columnar_scan and
pgc_uses_row_fetch are shared by many suites, and eighteen suite files changed.
The selftest cannot exercise them; the two `suites (PG N)` jobs can. Local:
selftest exit 0, 553 checks, 0 failures, shellcheck rc=0 over test/, selftest/
and bench/.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Unuuvh3fRR67SceiGpfeeK
(cherry picked from commit 0d49e20)

…nchor (#916)

Both reported by OffgridwithJD against 0ce14d7, and both are the shapes this
change exists to refuse, committed inside it.

THE PARTITION ARM WAS AN IDENTITY. `_reg` was incremented in the same loop body
as the three buckets, so their sum equalled it for ANY reader. Proven rather than
argued: it passes with an always-yes reader and with an always-no reader alike,
and adding the absent bucket did not change that. A total derived from the loop
that produces the buckets cannot fail, which is the note I had just re-worded two
functions away for the same reason.

The population is now counted by a SECOND ROUTE -- the runner's own --list-suites
-- so the arm fails when the classification does not see every registered suite:
a future `continue`, a read that drops a line, a list that changes between the two
reads.

And it now says what it is. It is a COVERAGE check, not a check on the reader's
correctness. The two arms below it, which require both buckets to be occupied, are
what catch a reader that answers the same way for everything. Overrating it is how
it survived as an identity.

NOTHING EXERCISED THE ^ ANCHOR. pgc_log_shows_accounting's comment claims
"anchored and fully shaped, so the word appearing in a suite's own prose cannot
satisfy it", and the prose fixture is refused by the regex SHAPE rather than by
the anchor -- so removing ^ from the reader left every arm in both harnesses
green. The indented fixtures added earlier are for the DECLARATION reader and do
not reach this one.

The distinguishing input is a well-formed accounting line that does not start its
line. Measured against a twin with the anchor removed, mutation asserted applied:

    indented line, real reader (anchored)     no
    indented line, twin reader (unanchored)   yes
    control, line-start, real reader          yes

Inert on real data -- 0 non-line-start occurrences across 246 PG17 logs and 244
PG18 -- so this closes a coverage gap rather than a live defect.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Unuuvh3fRR67SceiGpfeeK

Check results were prose. `check`, `check_num` and `check_text` printed PASS or
FAIL and nothing else, so proving that a mutation reddened one NAMED check meant
grepping text -- which is how every mutation proof in this repository is currently
made, a person reading `FAIL  <name>` out of a log and retyping it.

That is also how a reverted guard once reported plain green while the check count
fell from 190 to 186: the suite passed, and the only evidence anything had changed
was a number nobody was comparing.

THE FIX IS NOT A SECOND EMITTER. A second source of truth for how many checks ran
is the defect this family of issues exists to close. lib.sh had ELEVEN places that
bumped PGC_CHECKS, each with its own outcome line beside it -- eleven chances to
add a twelfth and forget the line, which is exactly what projections.sh's
expect_fail did with ten call sites for as long as it existed.

So counting a check and recording it are ONE operation, pgc_record. A helper
cannot report an outcome without being counted, and cannot be counted without
reporting one, because no code path does either alone. `checks run: N` and the N
record lines are the same increment seen twice. Eleven sites became one, and the
arm that holds it is structural: lib.sh may bump PGC_CHECKS in exactly one place,
and that place must be pgc_record.

The record is tab separated -- suite, name, verdict, reason -- so a check name
containing spaces survives. The reason carries #915's REASON_CODE, which is what
makes this more than a reformat: an unrunnable check is distinguishable from a
passing one without parsing prose. A verdict pgc_record does not recognise is
recorded as a FAIL rather than dropped, because dropping it would leave PGC_CHECKS
bumped with no outcome recorded -- the reconciliation pgc_summary already refuses.

THE HUMAN LINES DID NOT MOVE. DISPLAY is passed to pgc_record whole rather than
composed inside it, and both harnesses pin the exact strings for check, check_text,
check_num and check_unrunnable. 3,762 call sites, with suites, selftests and CI all
grepping `^PASS` and `^FAIL`, is far past what a careful refactor can be trusted on.

PGC_SUITE is resolved once at load rather than per check: pgc_record runs at every
one of those call sites, and a basename fork at each is 3,762 forks a suite does
not need.

The runner reconciles the two artifacts per suite: a log states `checks run: N` and
carries N records. That cannot fail by drifting, since one function does both, but
it can fail -- a suite killed mid-way, a truncated log, a helper that prints an
outcome without recording it. A log with no count at all never reached its summary,
which is a different fault from a miscount and is reported as one rather than
reading as a clean reconciliation.

Stacked on #916, which supplies pgc_log_shows_accounting: only a suite that reached
its summary has a count to reconcile against.

Evidence: selftest exit 0, 535 checks, 535 records, 0 failures; 14 pytest tests;
shellcheck -S error rc=0; docs_style PASSED.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Unuuvh3fRR67SceiGpfeeK

…adable-results

# Conflicts:
#	test/run_all_versions.sh

…adable-results

… made the reconciler wrong (#917)

Both from OffgridwithJD's review, and the first is embarrassing in the specific
way that makes it worth writing down.

FOUR FORKS PER RECORD, IN THE FUNCTION THAT HOISTS PGC_SUITE OUT TO AVOID THEM.
pgc_record ran `$(printf ... | tr)` twice to blank tabs -- two subshells and two
tr processes -- at every one of 3,762 check sites, three lines below a comment
explaining that a basename fork per check is 3,762 forks a suite does not need.
Parameter expansion does it free. Measured on an idle box, 2,000 calls, output
identical on every input including a real tab, a leading tab and a trailing one:

    printf | tr in $( )    3.1577 ms per call
    ${var//tab/ }          0.0096 ms per call
    ratio                       331x
    across 3,762 checks    11.9 s of pure fork overhead against 36 ms

DIRECT COUNTER WRITES MADE THE RECONCILER WRONG, and the two rules disagreed.
Selftest 320 blessed a direct PGC_CHECKS bump that records an outcome nearby;
pgc_reconcile_records requires a RESULT line. Thirteen sites across ten suites
took the first path, so on a failure the reconciler added
`records=N but the log states checks run: N+1` on top of the real failure.

The fix is not to soften the reconciler. Counting a check and recording it are one
operation, which is this change's whole argument, and a direct write is a check
counted with nothing recorded -- the hole that argument cannot have. All thirteen
now call pgc_fail, which lib.sh's own header already calls "the only supported
way to add a check from outside this file". Selftest 320 gains the stronger rule
that was not satisfiable until now: no suite using lib.sh's accounting writes
PGC_CHECKS directly. bench_guards keeps its own counter under the same name and
never sources lib.sh, and is exempt by measurement rather than by name.

TWO OF MY OWN ARMS WERE WRONG ON THE WAY THROUGH, both caught by running them.

The old premise required FIVE direct writes to EXIST, which is a premise about
the corpus rather than about the sweep -- so converting the thirteen turned it red
for exactly the reason the change is for. A premise that fails when the thing it
guards is fixed is the wrong premise. It now asserts the sweep read something, and
both rules are proven on fixtures: the original flags a bump with no outcome and
allows one with an outcome, the stronger one flags that same allowed bump, and
both exempt a private counter. Two arms reading "[]" over a clean corpus are
satisfied by a sweep that classifies nothing.

And those fixtures, written out literally, made this file flag its own three
generator lines -- the same mistake selftest 080's control avoids by living in a
quoted heredoc. The bump is now assembled from the variable name.

Evidence: selftest exit 0, 581 checks, 581 records, 0 failures; 133 pytest passed
(the 35 errors are /usr/local/pg18a absent on this host, identical on main);
shellcheck rc=0 across test/, selftest/ and bench/.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Unuuvh3fRR67SceiGpfeeK

jd's standing rule is that no PR ships without its CHANGELOG and docs in the same
PR. `git diff --name-only main HEAD -- CHANGELOG.md` was 0 files for this branch,
which OffgridwithJD caught: between this and #918 the stack adds pgc_record, a
machine-readable RESULT format across every call site, a fourth counted outcome,
three selftest parts and two tracked data files, and none of it was recorded.

The entry covers what a reader of the release notes needs: why one counter rather
than eleven, why the record names the part and not just the suite, why a skipped
wall-clock check is counted, and what the matrix now reconciles.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Unuuvh3fRR67SceiGpfeeK